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ABOUT THIS REPORT 


Education should be a citywide concern, much like public safety and public health. We designed this research project for 
city leaders who want to evaluate how well all the schools in their city— whether they are district- or charter-governed— are 
serving their city’s children and how their city’s schools compare to those in other cities. To our knowledge, this is the first 
time a cross-sector, citywide analysis of public education has been made available. 

What started out as a technical task— pulling together publicly available data to develop a set of “indicators” city leaders 
could use to measure school performance and equity— ended up raising serious questions about the health of our urban 
schools. It also uncovered places where progress in urban education is being made. 

For each indicator in the report we’ve highlighted a few cases that caught our eye; some are areas of concern, others are 
bright spots. We hope city leaders and others will use these prompts and our online data at crpe.org to frame their own 
questions and develop their own solutions. 
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Through research and policy analysis, CRPE seeks ways to make public education more effective, especially for America’s 
disadvantaged students. We help redesign governance, oversight, and dynamic education delivery systems to make it 
possible for great educators to do their best work with students and to create a wide range of high-quality public school 
options for families. Our work emphasizes evidence over posture and confronts hard truths. We search outside the traditional 
boundaries of public education to find pragmatic, equitable, and promising approaches to address the complex challenges 
facing public education. Our goal is to create new possibilities for the parents, educators, and public officials who strive to 
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FOREWORD 


Public education has historically provided a pathway out of poverty for our most vulnerable youth. However, that pathway has 
been eroded. 

City leaders know this better than anyone. In April 2015, Politico Magazine reported results from a survey of 20 mayors 
across the country about the challenges facing their cities. The mayors listed public education second only to “deteriorating 
infrastructure” as their biggest challenge. 

The future of our country depends on our youth. Yet our public education system, designed more than 100 years ago, faces 
significant new challenges in preparing all students for success in career and civic society. Today, students of color and 
students from low-income families make up the majority of public school students in the U.S. 1 Urban leaders are struggling 
to provide hope and opportunity for these youth, many of whom live in cities and neighborhoods where the schools are 
ineffective, jobs are scarce, and poverty is widespread. 

For years we have seen evidence of persistent achievement and opportunity gaps between racial and socio-economic student 
groups. Some have concluded that poverty and racial inequities are conditions that schools cannot overcome. 

This report shows that conclusion is, at least, premature. It shows that while the inequities are profound, cities can create schools 
that serve all students well. It shows that educators can find ways to give more students access to challenging curriculum and a 
pathway to college and career. The question before us is how we can create those opportunities for all students. 

Measuring Up: Educational Improvement and Opportunity in 50 Cities speaks to those who are concerned about the overall 
health of America’s urban schools. It provides the first comprehensive view of all schools in a city, whether district-run or 
charter. 2 We selected the cities based on their size and because they reflect the complexity of urban public education today, 
where a single school district is often not the only education game in town. We went beyond test scores, using a variety of 
publicly available state and federal data to measure school system health and educational opportunity for students from low- 
income households and students of color. 


1. For example, see William J. Hussar & Tabitha M. Bailey, Projections of Education Statistics to 2022. 41st ed. (Washington, DC: National Center 
for Education Statistics, U.S. Department of Education, 2014); and Lesli A. Maxwell, “ US School Enrollment Hits Majority-Minority Milestone. ” 
Education Week, Aug. 19, 2014. 

2. The National Assessment of Education Progress, Trial Urban Assessments (NAEP-TUDA) provides limited information on academic 
achievement based on standardized test results in core subjects. It is limited to 21 urban districts and does not include charter schools. The 
Center for Research on Education Outcomes (CREDO) has published reports on charter school performance relative to traditional public 
schools. The Brookings Institution's Education Choice and Competition Index scores large school districts based on choice-related policy and 
practice but does not assess citywide opportunity and improvement. 
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Citywide Indicators 

How well are the city’s schools doing overall? 

• School-level gains in math and reading proficiency relative to state performance 

• High school graduation rates 

• Share of students enrolled in “beat the odds” schools 

• Share of schools stuck in bottom 5 percent of the state based on proficiency 
rates that stay there for three years running 

How well are they doing for students from low-income households and students 
of color? 

• Enrollment in highest- and lowest-scoring elementary and middle schools 

• Proficiency gaps for students eligible for free and reduced-price lunch (FRL) 

• Advanced math course-taking 

• ACT/SAT test-taking 

• Out-of-school suspensions 


Looking across all the cities, 
we see four themes: 

INEQUITY IN PUBLIC EDUCATION, THOUGH WIDESPREAD, IS NOT 
INEVITABLE. 

Results in many cities offer optimism that things can be better: 

• In Washington D.C., FRL-eligible students enroll in top-scoring schools at higher rates than their more advantaged peers. 

• In 20 of the 50 cities, black students take the ACT/SAT at the same or better rates than white students. In Santa Ana, 
California, a 6-percentage point ACT/SAT test-taking gap favors black students. 3 

• A handful of cities appear to be successful at either fixing or closing their lowest-performing schools: In New Orleans and 
Memphis, none of the schools that performed in the bottom 5 percent in the first year of our data (for reading and math) 
stayed at that level for three consecutive years. 


BUT PERFORMANCE IN MOST CITIES IS STILL FLAT. 

In the three most recent years of available data: 

• Less than a third of the cities we studied made proficiency gains relative to their state’s performance (only 12 made 
overall gains in math proficiency and only 14 made gains in reading). 


3. Improving access to the ACT/SAT is important, but a recent report from ACT and the United Negro College Fund highlights the gaps 
between ACT results for black students and those of other students, showing that access to these tests is not enough to improve college 
readiness rates among black students. See The Condition of College and Career Readiness 2014: African American Students. 
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• Eight cities are actually falling behind their earlier performance in math, in reading, or in both subjects relative to their 
state’s performance. 

• One in four students do not graduate high school in four years. 

• Large shares of schools (40 percent across all these cities) that were in the bottom 5 percent of all schools statewide in 
year one stayed stuck there for three years running. 


POOR AND MINORITY STUDENTS STILL FACE STAGGERING 
ACADEMIC INEQUITIES. 


• FRL-eligible students score lower than other students in nearly every city. The gaps are especially large in some cities 
(Denver, Cleveland, and Raleigh) but smaller in others (Santa Ana, Detroit, and Los Angeles). 

• With few exceptions, FRL-eligible students and students of color are less likely than white students in the same cities to 
enroll in high-scoring elementary and middle schools, take advanced math courses, and take the ACT/SAT. 

• In every city, some schools “beat” their demographic odds, but on average, only 8 percent of students in the cities we 
studied are enrolled in schools (district or charter) that got better results than schools with similar student demographics 
in the state. 


THE PICTURE IS ESPECIALLY BLEAK FOR BLACK STUDENTS. 


• In Newark— where a majority of students are black— only 6 percent of black students enrolled in a top-scoring elementary 
or middle school (in math) compared to 85 percent of white students. 

• In every city we studied except for Baton Rouge, black students are much more likely to be suspended than white students. 

• In Portland, Oregon, and Seattle, black students were at least four times more likely than white students to attend a 
school scoring in the cities’ bottom 20 percent in math. 

How to use this report: 

We hope this report and the online data that go with it will serve as a catalyst for cities to take a comprehensive look at their 
schools, ask tough questions, and find other cities to turn to for inspiration. We did not measure outcomes against specific 
improvement strategies. However, city leaders looking for solutions can use our analysis to identify and learn more from cities 
that are ahead of the curve on certain indicators. City leaders might ask, for example, 

• How have New Orleans and other cities managed to improve or replace so many of their lowest- 
performing schools? 

• What is happening in cities like Memphis and Chicago— where black students participate in advanced 
courses and the SAT at high rates? 

• Why do some cities, like Newark and Cincinnati, have high numbers of schools that “beat the odds” by 
performing better than schools with similar demographics? 

• What accounts for the favorable discipline outcomes in cities like Baton Rouge, the only city we studied 
where black students are not suspended at higher rates than white students? Or Los Angeles, where 
overall suspension rates are low and Hispanic students are less likely to be suspended than white students? 
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The implications of this report should serve as a call to action. In order for America’s cities to move forward and for all 
of our youth to have real opportunities to learn, urban public education needs to focus on more than just incremental 
improvement. 

As a start, we should acknowledge and address the systemic reasons that academic segregation occurs so blatantly in 
our urban public schools. For years, research has documented within-district inequities in funding and access to quality 
teachers and other resources. CRPE and others have documented how state funding, district policies, union contracts, and 
neighborhood assignment provisions can reinforce educational inequity . 4 

Rather than be distracted by dogfights over Common Core, testing, choice, teacher evaluations, charter schools, and other 
policy debates, our city school system leaders need to aggressively hunt for and be open to new solutions, and respond 
quickly and meaningfully to shifting demographics and other challenges. 

The varied results in this report suggest that no single model for providing or governing schools— district operation, 
chartering, or vouchers— has been a sure solution to address the needs of urban students. What we can say, however, is that 
given the enormity of the challenges represented in this study, no city should close off any possible source of good schools, 
or favor its existing schools over options that might create new opportunities for children. Emerging approaches to school 
governance and whole community change, starting with early childhood, should be tried more broadly and aggressively . 5 

In the meantime, there are things every city can do immediately to overcome the lack of opportunity facing too many low- 
income students and students of color. They can: 

• Find ways to improve or replace the bottom 5 to 10 percent of schools with better options and move 
students into more effective teaching and learning environments. Cities like New Orleans have done this by 
having clear and tightly enforced accountability standards and by investing in effective new schools that can replace low- 
performing schools. 

• Insist that all students can and should have access to advanced placement and other college-prep 
coursework. Many of the cities we studied, like Cleveland and Denver, are investing in innovative, technology-driven 
school models to make that access possible for all students. 

• Make a frontal attack on overly aggressive discipline policies. Some cities, like Washington, D.C., have started 
publishing suspension and expulsion rates citywide and asking schools to voluntarily reduce their rates. Safe and orderly 
schools are necessary, but high-performing schools can find ways to maintain order without overly severe consequences 
for students. 


4. For example, see Natasha Ushomirsky and David Williams, Funding Gaos 2015: Too Many States Still Spend Less on Educating Students Who 
Need the Most (Washington, DC: The Education Trust, 2015); Dan Goldhaber, Lesley Lavery, and Roddy Theobald, “Uneven Playing Field? 
Assessing the Teacher Quality Gap Between Advantaged and Disadvantaged Students.” Educational Researcher 44 (no. 5): 293-307; Joshua 
M. Cowen and Katharine 0. Strunk, “The Impact of Teachers' Unions on Educational Outcomes: What We Know and What We Need to Learn.” 
Economics of Education Review (March 2015); Annette Lareau and Kimberly Goyette, ed., Choosing Homes. Choosing Schools (New York, NY: 
Russell Sage Foundation, 2014); Marguerite Roza and Paul Hill, How Within-District Spending Inequities Help Some Schools Fail (Seattle. WA: 
Center on Reinventing Public Education, 2004). 

5. See, for example, Paul Hill, Christine Campbell, and James Harvey, It Takes a City: Getting Serious About Urban School Reform (Washington, 
DC: Brookings Institution Press, 2000); Paul Hill, Christine Campbell, and Betheny Gross. Strife and Progress: Portfolio Strategies for Managing 
Urban Schools (Washington, DC: Brookings Institution Press, 2013); and Paul Hill and Ashley Jochim, A Democratic Constitution for Public 
Education (Chicago. IL: University of Chicago Press, 2014). 
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At the same time, doing better will require long-term commitment to a search for more effective strategies. We urge cities to: 


• Double down on bold, evidence-based solutions. Cities must be open to any promising school— district or 
charter— if it opens up new possibilities. City leaders must address their weaknesses head on and search widely for new 
solutions. 

• Recognize that the hard work ahead cannot be the work of schools alone. Cities like Memphis and New Orleans 
that are radically redesigning their schools and school systems are seeing results, but even these efforts need continued, 
coordinated support from teacher preparation programs and social and health services. They also need city and state 
leaders to support them when they have to make hard decisions— new leadership, turnaround, etc.— about failing schools. 

CRPE has, over the last 20 years, been developing new citywide governance frameworks and support systems. 6 We will 
continue to develop and test new approaches and track these cities’ progress in coming years. 

America is at a profound moment of social struggle. More children grow up in poverty, more young people end up 
incarcerated, persistent racial bias holds back opportunity. School improvement cannot wait for us to solve poverty or racial 
injustice. We can create great school options now for young people that can help to mitigate these other social challenges. 

We hope this report will be both a source of urgency and a source of hope. Results are discouraging. But what should make 
us both angry and hopeful is that there is evidence that things don’t have to be this way. 

We can and we must do better. We cannot improve our cities without improving our schools. 


Robin Lake 

Director, Center on Reinventing Public Education 


6. See crpe.org for our research, proposals, and tools for city leaders. 
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INTRODUCTION 


In the winter of 2015, Armen Hratchian, vice president of Excellent Schools Detroit, was preparing to talk to a group of 
stakeholders about Detroit’s troubled public school system. He and other leaders knew that the city wouldn’t fully rebound 
from its bankruptcy unless it had strong public schools, and they thought the time was right for the community to have a 
conversation about how Detroit could steer its schools toward a brighter future. 1 

To help set the stage for that conversation, Hratchian wanted to look at how Detroit’s schools were doing compared 
to schools in other cities. He thought the comparison would help people benchmark Detroit’s performance and better 
understand the challenges it faced. But making those comparisons was easier said than done. 

Hratchian’s first challenge was summarizing the performance of schools citywide. Getting a holistic view of the city’s schools 
was complicated because Detroit had a patchwork school system made up of Detroit Public Schools (DPS), Michigan’s 
Educational Achievement Authority, and a surging charter school sector. Luckily, Hratchian and his colleagues had worked 
hard to develop a novel citvwide scorecard that covered schools in all three systems, allowing them to show how all of the 
city’s public schools were doing, regardless of who oversaw them. 

But when it came to comparing Detroit to other cities, Hratchian wasn’t sure where to turn. If other cities happened to have 
patchwork governance systems like Detroit's, he had no way of capturing their citywide performance, since few had citywide 
scorecards. 

Other benchmarks fell short, too. Hratchian respected The Center for Research on Education Outcome’s charter school 
studies (they showed that Detroit’s charter schools outperformed DPS), but he wondered how the city’s charter schools did 
compared to those in places like Washington, D.C., or New Orleans, where he suspected the bar was higher. He also knew 
that Detroit scored at the bottom of The Nation's Report Card— the National Assessment of Educational Progress— but those 
results only included DPS and left out the city’s large charter school sector. 

Without a single way to measure all public schools citywide and compare Detroit to other cities, Hratchian was having a hard 
time putting Detroit’s overall performance in context. And that made understanding the challenges the city faced and finding 
ideas, inspiration— and cautions— about big-city school improvement harder than it should have been. 

At CRPE, we have spent the last year making the case that Hratchian and other civic and education leaders need to start 
viewing public education as a citywide concern, just as they do related issues like public health, economic development, 
and public safety. But taking that perspective is difficult when leaders don’t have a way of gauging the health of public 
education citywide. 2 Especially in cities like Detroit, where education governance is fragmented, city leaders can be at a 
loss to understand whether their public schools are getting better or worse and how they compare to schools in other cities. 
That’s a problem. As urban public education becomes more diverse and complex with district, charter, and— sometimes— state 
systems co-existing, city leaders need a handle on how all public schools are doing if they want to mobilize political action 
to address cross-cutting challenges that affect families and schools, from uneven school quality to unequal access to high- 
performing schools. 

This report offers a jumping off point for leaders interested in benchmarking and taking responsibility for the quality of not 
just some of the public schools in a city but of all of them. It does so by describing public schools in 50 mid- and large-sized 
cities; places where, like Detroit, a single school district is often no longer the only game in town. 


CRPE.ORG | 6 




MEASURING UP 


OUR APPROACH 


This report is about entire cities, not school districts. 3 Using a combination of federal and state data, we examined information 
on all of the public schools— regardless of the school district they belong to or who oversees them— in a sample of 50 cities 
that reflect the complexity of urban public education today. 

Many of the cities in this report also reflect the country’s changing demographics. Students of color and those from low- 
income households now make up the majority of public school students nationwide. By 2050, people of color will make up 
the majority of the U.S. population. 4 These cities are a window into the growing diversity of the United States (see Appendix 
A for U.S. Census and enrollment data from each city). 


Changing Student Demographics 

Percentage distribution of public school enrollment by race/ethnicity 1995-2013 
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Source: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "State Nonfiscal Survey of Public 
Elementary and Secondary Education, ’’ 1995-96 through 2011-12 and National Elementary and Secondary Enrollment Projection Model, 1972 
through 2023. (This table was prepared December 2013.) 


How We Selected the Cities 

This report intentionally focuses on cities with complex education landscapes, where multiple agencies oversee public 
schools and enrollments are spread across a variety of school types. To select the cities, we started with lists from the U.S. 
Department of Education’s Common Core of Data (CCD) and Private School Universe Survey (PSS) of every public and 
private school in the United States that was open during the 2011-12 school year— just over 132,000 schools. Though we don’t 
measure them in the indicators, we included private schools in our selection criteria to capture the entire scope of schools 
available in urban America today. 5 

The CCD and PSS include a lot of information about the nation’s public and private schools, including each school’s 
geographic coordinates. Using those coordinates, we located each school within a municipal boundary, based on data from 
the U.S. Census. 6 We didn’t look at unusual types of schools— for example, juvenile justice centers or schools for the blind; we 
only included regular public and private schools in our list. 
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After linking each school to its municipality, we used information on enrollment and school type to get a sense of how 
enrollments were spread across traditional district-run, charter, and private schools in different municipalities. To capture 
the most varied environments, we picked the 50 cities with the largest total enrollments that were also the most widely 
distributed across the sectors. 7 


The 50 Cities 

The resulting list includes an interesting mix of cities (Figure 1). It includes cities known for educational reform, like Denver, 
New Orleans, Washington, D.C., and New York City, but also includes cities that make few headlines beyond the local news 
such as Tampa, Tucson, and Albuquerque. Since we included private school enrollments, some cities make the list simply 
because they have large private sectors (e.g., Seattle, where 22 percent of families chose private schools in the 2011-12 school 
year). Finally, the list captures a range of city sizes. In addition to New York, we’ve included mega-cities like Los Angeles and 
Flouston, as well as mid-sized cities like St. Paul (MN), Baton Rouge, and Newark. 


Figure 1. 50 City Sample 
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Enrollments in district-run schools, charter schools, and private schools vary widely across the 50 cities (Figure 2). 8 In ten 
cities, a third of students enrolled in schools outside of the traditional district sector. These cities fell into two types: those 
where the majority of non-district enrollments were in private schools (San Francisco, Toledo, Milwaukee, and Cincinnati) and 
those where the majority of non-district enrollments were in charter schools (Cleveland, Columbus, Detroit, New Orleans, 
Philadelphia, and Washington, D.C.). In the remaining 40 cities, the majority of students enrolled in district schools. 
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Figure 2. Enrollment Share by Sector, 2011-2012 
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A patchwork of school districts and charter authorizers oversee public schools in the 50 cities, as shown in Figure 3. Each 
rectangle in Figure 3 represents citywide K-12 enrollment for 2011-12. Within each rectangle, the blue boxes represent 
enrollments overseen by school districts, the green boxes represent enrollments in charter schools that were authorized by 
traditional school districts, the orange boxes represent enrollments overseen by independent charter authorizers, and the 
gray boxes represent private school enrollments. 

Thirty of the 50 cities have multiple traditional school districts and 34 have multiple charter school authorizers. It’s critical to 
keep this pluralism in mind while reading this report. 
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Figure 3. Educational Oversight Agencies and Private Sectors, 2011-2012 
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Definitions and Sources 

Figures 2 and 3 use enrollment counts from the U.S. Department of Education’s Common Core of Data (CCD) and 
Private School Survey (PSS) from the 2011-12 school year, the most recent year of the PSS available at the time 
we wrote the report. Figure 3 links these enrollment counts with information on oversight agencies (districts and 
charter authorizers). To identify oversight agencies associated with traditional public schools, we used the school 
districts listed for individual schools in the CCD. To identify oversight agencies associated with charter schools, 
we used data from the National Association of Public Charter Schools from 2011-12 that list each charter school’s 
charter school authorizes 

Source: National Center for Education Statistics, Common Core of Data, Public Elementary/Secondary School Universe Survey ; 
Private School Survey, 2011-12 ; National Association of Public Charter Schools, Public Charter Schools Dashboard, 2011-12. 
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MEASURING UP 


The Advantages and Limitations of Publicly Available School-Level Data 

In addition to the CCD, we rely on two other major data sources for the indicators: school-level files from the Office for 
Civil Rights (OCR), Civil Rights Data Collection (CRDC). We use high school graduation data from the U.S. Department of 
Education’s EdFacts Data Files. We used publicly available, school-level data because they allowed us to create measures 
across all (or most) of the cities in the sample. Using aggregate school-level data also makes our work transparent and 
reproducible. Although pulling together the data for this report was time consuming, obtaining and preparing student-level 
data for every district-run and charter school in all 50 cities and their states would have been even more difficult, if not 
impossible. 

Of course, the data also have several limitations. Because of the time it takes for states to release data, for example, the 
results are necessarily lagged and do not reflect the most recent developments in any of the cities. In addition, the states in 
the report do not all provide data that cover the same years. For the majority of the states, the three most recently available 
years of data cover 2012-2014, but for some states, publicly available data doesn’t go beyond 2012 (see Appendix A for the 
years available for each city’s state). 

One of the biggest problems with the state data is that most states provide school-level proficiency rates in their publicly 
available data rather than continuous measures of student achievement (see our data inventory in Appendix A). This 
creates two issues. First, it makes it impossible to directly compare performance levels across cities because expectations 
for proficiency vary widely by state. 9 Since we can’t directly compare proficiency rates across states, we built the indicators 
around relative measures of performance, like the share of FRL-eligible students in a city who are enrolled in its top-scoring 
schools. 10 Second, proficiency rates ignore important information because they are binary: a student is or is not proficient. 

As a result, two schools can have similar proficiency rates but different underlying proficiency profiles. Imagine, for example, 
School A, where most students are just over the proficiency line, compared to School B, where a number of students are far 
above the proficiency line and a similar number are far below. If School B’s scores averaged out to the same proficiency rate 
as School A, they would look the same despite their underlying variation. 

Our second major data source, the OCR, also has some important limitations. Although the OCR data are useful for looking 
across cities, they suffer from shortcomings associated with survey data (e.g., respondents interpreting questions differently). 
In addition, the way all of these data sources identify groups of students relies on crude measures, such as using “free and 
reduced-price lunch” (FRL) eligibility as a poverty measure, or using “Flispanic” to identify a population of students that is far 
more heterogeneous than a single label implies. 


Why We Didn’t Use NAEP Scores to Compare Performance 

Assessing school performance is difficult. It involves measurement issues but also 
deeper questions about the values and purposes of public education . 11 These challenges 
notwithstanding, policymakers and leaders need to gauge how well students are learning 
and, for better or for worse, they currently do so using student performance on standardized 
tests. As we noted earlier, making cross-city comparisons of proficiency rates is impossible 
because states define proficiency differently. Researchers have, however, mapped state 
proficiency standards onto a common scale defined by the National Assessment of 
Educational Progress (NAEP) to conduct state-to-state comparisons . 12 We explored using 
the same discounting procedure at the city level for this project, but the results were 
disappointing. When we compared our NAEP-discounted city rankings to rankings based 
on the NAEP TUDA (for district schools only), the results did not line up. We also compared 
NAEP-adjusted city rankings to rankings based on scale scores for cities where scale scores 
were available, but again, the results pointed in different directions. In the end, the underlying 
performance distributions in the cities and their states are probably too different to naively 
apply the NAEP discount; unfortunately, we could not assess that possibility directly because 
of the limitations of the state data. 
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MEASURING UP 


THE INDICATORS 


Our indicators sought to answer two overarching questions: 

• How well are each city’s schools doing overall? 

• How well are they doing for students from low-income households and students of color, who make up a majority of the 
public school population? 

Using the three most recent years of available data for each city (see Appendix A for years covered), we developed nine 
indicators that address these questions. 


Citywide Indicators 

How well are the city’s schools doing overall? 

• School-level gains in math and reading proficiency relative to 
state performance 

• High school graduation rates 

• Share of students enrolled in “beat the odds” schools 

• Share of schools stuck in bottom 5 percent of the state based 
on proficiency rates that stay there for three years running 

How well are they doing for students from low-income households 
and students of color? 

• Enrollment in highest- and lowest-scoring elementary and 
middle schools 

• Proficiency gaps for students eligible for FRL 

• Advanced math course-taking 

• ACT/SAT test-taking 

• Out-of-school suspensions 


Many cities look successful on a few indicators but none look successful across all, or even most, of them. Our results suggest 
how difficult it is to ensure both quality and equity in urban education. 
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MEASURING UP 


PART I: ACADEMIC ACHIEVEMENT AND IMPROVEMENT 


To gauge citywide academic achievement and improvement in our 50-city sample, this 
section covers four measures: 

1. School-level gains in math and reading proficiency relative to state performance. 

2. High school graduation rates. 

3. Share of students enrolled in “beat the odds” schools. 

4. Share of schools stuck in bottom 5 percent of the state based on proficiency rates that stay there for three years running. 
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MEASURING UP 


INDICATOR | Citywide Gains in Math and Reading Proficiency 


Less than a third of the cities 
made gains in math or reading 
proficiency over the three most 
recent years of data relative to 
their state’s performance. 


Cities with proficiency gains in both math and reading for the 
three most recent years of available data relative to their state’s 
performance: 


Baton Rouge Indianapolis Raleigh (NC) 

Boston Los Angeles Tampa 

Denver New York City 
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MEASURING UP 


Figure 4. Citywide Gains in Math and Reading Proficiency Relative to State Performance 
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Definitions and Sources 

Figure 4 shows whether cities made school-level gains in proficiency over the three most recent years of available 
data (See Appendix A for data range for each city’s state). The gains are in standard deviation units and the solid 
bars in the chart show statistically significant gains (p <0.05). We standardized school proficiency rates by state 
and year to try to account for differences in state proficiency standards and year-to-year shifts in state assessments 
or standards. To show gains, a city’s schools had to improve relative to its state’s performance. We also adjusted the 
results to account for the different mix of students served by each school; to make those demographic adjustments 
we used data from the CCD in our calculations because demographic data in state-level files was inconsistent and 
incomplete (more details on how we estimated the school-level gains are available in Appendix B). 

Sources: State Agency School Performance Files (see Appendix A for years/variables); National Center for Education Statistics, 
Common Core of Data, Public Elementary/Secondary School Universe Surveys 2009-2014. 
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MEASURING UP 


INDICATOR | High School Graduation Rates 


The average high school 
graduation rate across the cities 
was 75%. One in four ninth 
graders in 2009 didn’t graduate 
from high school in four years. 


90% of students graduated 
from high school in four 




graduated from high school 
in four years. 


w 
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MEASURING UP 


Figure 5. High School Graduation Rates for Class of 2013 
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Definitions and Sources 

We estimated graduation rates in Figure 5 using 9th grade cohorts from the 2009-10 school year and a four-year 
graduation rate. We took the number of those 9th graders who earned a regular high school diploma by 2012- 
13 and divided it by the number of first-time 9th graders in the starting cohort (fall 2009), plus students who 
transferred in, minus students who transferred out, emigrated, or died between 2009 and 2013. When the data 
provided a numerical range for a school’s rate, we used the mid-point (e.g., if a school’s rate was given as between 
50 and 54 percent, we recoded it as 52.5 percent). 

Sources: The EDFacts Initiative, U.S. Department of Education, Assessment and Adjusted Cohort Graduation Rates (ACGR) Data 
2009 - 2013 . 
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MEASURING UP 


INDICATOR | Share of Students in “Beating the Odds” Schools 


On average, only 8% of students 
are enrolled in schools that 
outperformed similar schools 
statewide over the last three years. 


In Newark and Cincinnati, about one out of three students 


enrolled in a school that outperformed similar schools statewide. 

w 
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MEASURING UP 


Figure 6. Average Share of Students in “Beat the Odds” Schools for Three Most 
Recent Years of Data 
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Definitions and Sources 

The measures in Figure 6 show the average share of students in a city enrolled in a school whose proficiency rates 
outpaced demographically similar schools elsewhere in the state, averaged over the three most recent years of 
available data (see Appendix A for data range for each city’s state). The results come from regression models that 
use information on all of the schools within a city’s state to predict the proportion of students in a school who 
score at or above proficiency based on the school’s student demographics and other school characteristics (we run 
separate models for math and reading; see Appendix B for the specific models). We used aggregate school-level 
proficiency data from the state files and student sub-group enrollment data from the CCD. The results (specifically 
the residuals) show the difference between a school’s actual proficiency rates and the rates we would predict, given 
the school’s student composition and other characteristics. We only identified schools as outperforming similar 
schools statewide when the probability that the residual is zero was less than 5 percent (p < .05 in a one-tailed 
significance test). The results can be interpreted as the proportion of students in a city who attended schools that 
outperformed similar schools in the state (see Appendix B for more detail). 

Sources: State Agency School Performance Files (see Appendix A for years/variables available); National Center for Education 
Statistics, Common Core of Data, Public Elementary/Secondary School Universe Surveys 2009-2014. 
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INDICATOR | Share of Schools Stuck in the Bottom 5% of State 


On average, about 40% of 

schools that were at the bottom 

5% of their state based on 
proficiency rates stayed stuck 
there for three years running. 


In five cities, none of the schools 
in the bottom 5% stayed stuck for 
all three years. 


Memphis (math and reading) 
New Orleans (math and reading) 
Santa Ana, CA (math) 
Washington, DC (math) 

Chandler, AZ (reading) 



in the first year of the data 
we collected. The majority 
remained stuck in the bottom 
5% for the next two years. 


Tables 1 and 2 show the percent of each city’s public schools (regardless of grade span) that scored in the lowest 5 
percent of schools statewide in math and reading in the first year of our data (Column 1), and the share of those same 
schools that remained in the bottom 5 percent three years later (Column 2). 
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MEASURING UP 


Table 1. Persistence of Schools in the Bottom 5 Percent From Year 1 to Year 3 in Math 


City 

Percent of schools in bottom 5% of state in year 1 

Percent of year 1 schools stuck for all 3 years 

Memphis, TN 

31.6 

0.0 

New Orleans, LA 

9.2 

0.0 

Santa Ana, CA 

1.5 

0.0 

Washington, DC 

5.5 

0.0 

Portland, OR 

6.1 

14.3 

Houston, TX 

7.6 

15.4 

San Diego, CA 

4.2 

16.7 

Cincinnati, OH 

15.5 

18.2 

Dallas, TX 

7.9 

18.2 

Baton Rouge, LA 

8.8 

20.0 

Tampa, FL 

6.2 

20.0 

San Francisco, CA 

12.0 

23.1 

Mesa, AZ 

3.6 

25.0 

Oakland, CA 

15.9 

25.0 

Tucson, AZ 

12.7 

25.0 

Chicago, IL 

16.8 

26.2 

Indianapolis, IN 

19.1 

29.0 

Wichita, KS 

19.1 

29.4 

Phoenix, AZ 

4.9 

33.3 

Detroit, Ml 

43.5 

34.4 

Louisville, KV 

12.5 

35.7 

St. Paul, MN 

15.3 

36.4 

Los Angeles, CA 

9.2 

37.5 

Nashville, TN 

6.2 

37.5 

Atlanta, GA 

20.3 

38.5 

New York, NY 

6.3 

39.2 

Fort Wayne, IN 

10.2 

40.0 

San Jose, CA 

2.4 

40.0 

Kansas City, MO 

18.2 

41.7 

Newark, NJ 

30.9 

42.9 

Minneapolis, MN 

27.0 

45.0 

Sacramento, CA 

3.5 

50.0 

Seattle, WA 

6.6 

50.0 

Albuquerque, NM 

7.5 

54.5 

Columbus, OH 

24.3 

54.8 

Jacksonville, FL 

3.6 

60.0 

Miami, FL 

9.4 

60.0 

Boston, MA 

12.7 

62.5 

Toledo, OH 

16.5 

64.3 

Milwaukee, Wl 

51.7 

64.9 

Baltimore, MD 

23.5 

65.1 

Denver, CO 

9.9 

66.7 

Raleigh, NC 

5.0 

66.7 

Stockton, CA 

7.2 

66.7 

Philadelphia, PA 

24.8 

67.6 

Colorado Springs, CO 

5.3 

71.4 

Pittsburgh, PA 

13.0 

71.4 

Cleveland, OH 

33.1 

82.2 

Chandler, AZ 

1.8 

100.0 

Chula Vista, CA 

3.4 

100.0 
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Table 2. Persistence of Schools in the Bottom 5 Percent From Year 1 to Year 3 in Reading 


City 

Percent of schools in bottom 5% of state in year 1 

Percent of year 1 schools stuck for all 3 years 

Chandler, AZ 

1.9 

0.0 

Memphis, TN 

30.3 

0.0 

New Orleans, LA 

4.6 

0.0 

Houston, TX 

8.7 

13.6 

Miami, FL 

11.3 

16.7 

Tucson, AZ 

7.8 

16.7 

Phoenix, AZ 

6.8 

20.0 

Washington, DC 

5.5 

22.2 

Fort Wayne, IN 

8.2 

25.0 

Tampa, FL 

9.9 

25.0 

Wichita, KS 

18.0 

25.0 

Atlanta, GA 

17.2 

27.3 

Indianapolis, IN 

20.4 

27.3 

Dallas, TX 

7.8 

28.6 

Pittsburgh, PA 

13.0 

28.6 

Sacramento, CA 

8.7 

30.0 

San Diego, CA 

3.5 

30.0 

Detroit, Ml 

37.4 

30.9 

Chicago, IL 

12.8 

32.8 

Chula Vista, CA 

5.1 

33.3 

Jacksonville, FL 

4.3 

33.3 

Mesa, AZ 

2.7 

33.3 

Raleigh, NC 

5.0 

33.3 

Santa Ana, CA 

4.6 

33.3 

Denver, CO 

16.8 

34.6 

Cincinnati, OH 

15.5 

36.4 

Nashville, TN 

8.6 

36.4 

New York, NY 

7.8 

40.7 

Portland, OR 

10.4 

41.7 

San Francisco, CA 

13.2 

42.9 

Newark, NJ 

23.5 

43.8 

Baton Rouge, LA 

15.8 

44.4 

Oakland, CA 

21.4 

44.4 

Seattle, WA 

9.9 

44.4 

Stockton, CA 

15.7 

46.2 

Los Angeles, CA 

8.4 

46.6 

Kansas City, MO 

24.2 

46.9 

St. Paul, MN 

11.1 

50.0 

Toledo, OH 

16.5 

50.0 

Columbus, OH 

24.4 

54.8 

Albuquerque, NM 

6.1 

55.6 

San Jose, CA 

3.3 

57.1 

Boston, MA 

18.3 

60.9 

Baltimore, MD 

25.7 

63.8 

Milwaukee, Wl 

51.3 

66.2 

Minneapolis, MN 

24.3 

66.7 

Philadelphia, PA 

28.2 

69.0 

Louisville, KY 

24.1 

74.1 

Cleveland, OH 

30.9 

81.0 

Colorado Springs, CO 

0.8 

100.0 
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Definitions and Sources 

Schools with student performance that falls into the lowest 5 percent statewide are often viewed as a state’s most 
troubled schools and a priority for improvement. When schools persist in this status for multiple years, it suggests 
that the local system— whether it’s one with a dominant school district, multiple school districts, or a mix of districts 
and charter schools— may not have the tools or mechanisms it needs to improve these schools. For the results in 
Tables 1 and 2, we used state aggregate school-level results to identify schools that ranked in the bottom 5 percent 
of their state in terms of math proficiency in each of three consecutive years of data (we did the same for reading). 
Then, using each school’s unique identifier, we looked to see how many of the schools that started in the bottom 
5 percent in year 1 remained in the bottom 5 percent in years 2 and 3. Schools that escaped “stuck” status may 
have improved their ranking, closed or been reconstituted, or improved their ranking for year two and returned to a 
bottom rank for year three. 

Sources: State Agency School Performance Files (see Appendix A for years/variables available); National Center for Education 
Statistics, Common Core of Data, Public Elementary/Secondary School Universe Surveys 2009-2014. 
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PART II: EDUCATIONAL OPPORTUNITY 


This section uses five measures to gauge educational opportunity for FRL-eligible students 
and students of color: 

1. Enrollment in highest- and lowest-scoring elementary and middle schools. 

2. Proficiency gaps for students eligible for FRL. 

3 Advanced math course-taking. 

4 ACT/SAT test-taking. 

5. Out-of-school suspensions. 
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INDICATOR | Enrollment in Highest- and Lowest-Scoring 
Elementary and Middle Schools 


In the vast majority of cities, students 
eligible for FRL and students of color 
are less likely to enroll in high-scoring 
elementary and middle schools 
than non-FRL students and white 
students, respectively. 


In Los Angeles, Hispanic 
students 


were almost seven 
times as likely as white 
students to be in a low- 
scoring elementary or middle 
school in math. 


In Phoenix, where the 
majority of students are 
Hispanic, 


Hispanic students 




were four times more likely 
than white students to be in 
a low-scoring elementary or 
middle school in math. 


Though blacks represent just 
over half of Newark’s student 
population, 


of the city’s black students 
attend a top-scoring school 
in math compared to 85% of 
white students. 



enrolled 

in top-scoring elementary 
and middle schools at 
higher rates than their more 
advantaged peers. 
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The figures in this section look at who attends the highest- and lowest-scoring elementary and middle schools in each 
city (top/bottom quintile in reading and math). The markers on the left side of each figure represent the share of 
each sub-population enrolled in a top- or bottom-scoring elementary or middle school in each city. To put the results 
in context, if sub-groups enrolled in schools at the same rates, the markers would be even and, by definition, clustered 
at 20 percent. The numbers on the right side of the figures compare the likelihood of different groups of students 
enrolling in the top- or bottom-scoring schools. Although student demographics, residence, and school performance 
are all clearly correlated, we nevertheless might still expect to see some variation across the cities due to the exercise 
of public school choice and the range of students and schools in the cities. 


Figure 7. Share of Students Enrolled in Top-Scoring Elementary and Middle Schools in 

Math by Income Status 


What percent of each group is in 
the city’s highest-performing schools? 


How many times more likely are non-FRL students than.. 
FRL students 
...to be in these schools? 
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MEASURING UP 


Figure 8. Share of Students Enrolled in Top-Scoring Elementary and Middle Schools in 

Reading by Income Status 
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MEASURING UP 


Figure 9. Share of Students Enrolled in Top-Scoring Elementary and Middle Schools in 

Math by Race/Ethnicity 
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MEASURING UP 


Figure 10. Share of Students Enrolled in Top-Scoring Elementary and Middle Schools in 

Reading by Race/Ethnicity 
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MEASURING UP 


Definitions and Sources 

The measures in figures 7 through 10 look at the enrollment of different groups of students in a city’s highest- 
scoring elementary and middle schools. We define a high-scoring school as one with proficiency rates in the top 20 
percent of schools citywide in the most recent year of available data (see Appendix A for data range for each city’s 
state). After using school-level proficiency rates to identify schools in the top 20 percent of each city’s performance 
distribution in the most recent year of available data, we then looked at enrollment rates in these schools by 
student sub-group to generate the measures in the chart. Although student demographics, residence, and school 
performance are all highly correlated, we nevertheless might expect to see variation across the cities due to 
exercise of public school choice and the range of students and schools in the cities. We excluded high schools from 
the analysis because of limitations in the high school data (e.g., too few tested grades and missing data). 

Sources: State Agency School Performance Files (see Appendix A for years/variables available); National Center for Education 
Statistics, Common Core of Data, Public Elementary/Secondary School Universe Survey. 


CRPE.ORG | 30 




MEASURING UP 


Figure 11. Share of Students Enrolled in Bottom-Scoring Elementary and Middle Schools in 

Math by Income Status 
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MEASURING UP 


Figure 12. Share of Students Enrolled in Bottom-Scoring Elementary and Middle 
Schools in Reading by Income Status 
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MEASURING UP 


Figure 13. Share of Students Enrolled in Bottom-Scoring Elementary and Middle 
Schools in Math by Race/Ethnicity 
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MEASURING UP 


Figure 14. Share of Students Enrolled in Bottom-Scoring Elementary and Middle 
Schools in Reading by Race/Ethnicity 
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MEASURING UP 


Definitions and Sources 

The measures in figures 11 through 14 look at the enrollment of different groups of students in a city’s lowest- 
scoring elementary and middle schools. We define a low-scoring school as one with proficiency rates in the bottom 
20 percent of schools citywide in the most recent year of available data (see Appendix A for data range for each 
city’s state). After using school-level proficiency rates to identify schools in the bottom 20 percent of each city’s 
performance distribution in the most recent year of available data, we then looked at enrollment rates in these 
schools by student sub-group to generate the measures in the chart. Although student demographics, residence, 
and school performance are all highly correlated, we nevertheless might expect to see variation across the cities 
due to exercise of public school choice and the range of students and schools in the cities. We excluded high 
schools from the analysis because of limitations in the high school data (e.g., too few grades and a large amount of 
missing data). 

Sources: State Agency School Performance Files (See Appendix A for years/variables available); National Center for Education 
Statistics, Common Core of Data, Public Elementary/Secondary School Universe Survey. 
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MEASURING UP 


INDICATOR | Proficiency Gaps for Students Eligible for Free 
and Reduced-Price Lunch 


On average, students eligible 
for FRL face double-digit 
achievement gaps— around 14 
percentage points— compared to 
other students. 
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MEASURING UP 


Figure 15. Average Achievement Gaps for Students Eligible for FRL Over the Three 
Most Recent Years of Available Data 


Math 

Santa Ana, CA 
Detroit, Ml 
Los Angeles, CA 
New York, NY 
Stockton, CA 
Dallas, TX 
Chula Vista, CA 
Houston, TX 
Baltimore, MD 
Sacramento, CA 
Newark, NJ 
San Francisco, CA 
Washington, DC 
Kansas City, MO 
Oakland, CA 
Indianapolis, IN 
Memphis, TN 
San Diego, CA 
Chicago, IL 
San Jose, CA 
Nashville, TN 
Fort Wayne, IN 
Albuquerque, NM 
Toledo, OH 
Colorado Springs, CO 
Cincinnati, OH 
Louisville, KY 
Milwaukee, Wl 
Atlanta, GA 
Columbus, OH 
Portland, OR 
Philadelphia, PA 
Seattle, WA 
Pittsburgh, PA 
Cleveland, OH 
Raleigh, NC 
Denver, CO 
Wichita, KS 
Tucson, AZ 
Tampa, FL 
St. Paul, MN 
Phoenix, AZ 
New Orleans, LA 
Minneapolis, MN 
Miami, FL 
Mesa, AZ 
Jacksonville, FL 
Chandler, AZ 
Boston, MA 
Baton Rouge, LA 


Santa Ana, CA 
Detroit, Ml 
Dallas, TX 
Los Angeles, CA 
Baltimore, MD 
Houston, TX 
Stockton, CA 
New York, NY 
Sacramento, CA 
Chula Vista, CA 
Washington, DC 
Kansas City, MO 
Oakland, CA 
Indianapolis, IN 
San Diego, CA 
Atlanta, GA 
San Francisco, CA 
Fort Wayne, IN 
Newark, NJ 
Chicago, IL 
Memphis, TN 
San Jose, CA 
Cincinnati, OH 
Toledo, OH 
Colorado Springs, CO 
Nashville, TN 
Albuquerque, NM 
Columbus, OH 
Louisville, KY 
Seattle, WA 
Pittsburgh, PA 
Milwaukee, Wl 
Philadelphia, PA 
Portland, OR 
Cleveland, OH 
Raleigh, NC 
Denver, CO 

* Wichita, KS 

* Tucson, AZ 

* Tampa, FL 

St. Paul, MN 
Phoenix, AZ 

New Orleans, LA 
Minneapolis, MN 
Miami, FL 
Mesa, AZ 
Jacksonville, FL 
Chandler, AZ 
Boston, MA 
Baton Rouge, LA 


Reading 



-45 -40 -35 -30 -25 -20 -15 -10 -5 0 5 

Percentage point difference between 
FRL and non-FRL students 



-45 -40 -35 -30 -25 -20 -15 -10 -5 0 5 

Percentage point difference between 
FRL and non-FRL students 


Definitions and Sources 

Figure 15 looks at gaps in proficiency rates between students eligible for FRL and non-FRL students averaged across 
the three most recent years of available data. Gaps are calculated by subtracting the proficiency rates of students 
who were not eligible for FRL from proficiency rates of students who qualified for FRL within a school, aggregated 
for each city. The results in Figure 15 are averaged across three years of data. Because of problems with missing data, 
we calculate the gap measure for only 37 of the 50 cities. When proficiency rates were only reported for students 
eligible for FRL, proficiency rates for the comparison group (non-low-income households) were calculated by using the 
relationships between overall proficiency, the low-income household proficiency, and their corresponding counts. 
Source: State Agency School Performance Files (see Appendix A for years/variables available). 
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MEASURING UP 


INDICATOR | Enrollment in Advanced Math Courses 


In 29 of the 50 cities, less than 

10% of high school students 

enrolled in advanced math 
classes in a given year. 


In Newark, Philadelphia, 
Memphis, Albuquerque, 
Detroit, Washington, D.C., 
and Cleveland, 


black students 


took advanced math at higher 
rates than white students. 


In Atlanta, Cincinnati, 
Columbus, Nashville, 
and Baltimore, 


Hispanic 


students took advanced 
math at higher rates than 
white students. 




w 
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Figure 16. Overall Share of Students Enrolled in Advanced Math and Percentage Point 
Gaps Between Groups, 2011-2012 


Overall 


Black v. white 


Hispanic v. white 


Jacksonville, FL 
Miami, FL 
Chicago, IL 
Tampa, FL 
Raleigh, NC 
Oakland, CA 
Minneapolis, MN 
New Orleans, LA 
San Francisco, CA 
Colorado Springs, CO 
Boston, MA 
Newark, NJ 
Atlanta, GA 
Chandler, AZ 
New York, NY 
Louisville, KY 
Cincinnati, OH 
Philadelphia, PA 
Houston, TX 
St. Paul, MN 
Memphis, TN 
Albuquerque, NM 
Dallas, TX 
Santa Ana, CA 
Pittsburgh, PA 
San Diego, CA 
Kansas City, MO 
Columbus, OH 
Seattle, WA 
San Jose, CA 
Portland, OR 
Chula Vista, CA 
Detroit, Ml 
Wichita, KS 
Nashville, TN 
Los Angeles, CA 
Milwaukee, Wl 
Denver, CO 
Indianapolis, IN 
Fort Wayne, IN 
Baton Rouge, LA 
Mesa, AZ 
Phoenix, AZ 
Baltimore, MD 
Toledo, OH 
Washington, DC 
Cleveland, OH 
Tucson, AZ 
Sacramento, CA 
Stockton, CA 




Definitions and Sources 

The measures in Figure 16 focus on the share of students taking advanced math courses in high school. The data come from 
the US Department of Education’s Office of Civil Rights’ CRDC survey. The CRDC defines advanced math courses to include 
topics like analytic geometry and trigonometry. 13 We calculated the rates by dividing the number of course/test takers in each 
high school by the total enrollment in that high school. We calculated sub-group rates by dividing the number of course/test 
takers in the subgroup in each high school by the total enrollment for that sub-group in that high school. Since most students 
take advanced math in 11th and 12th grade, we might roughly expect that somewhere between 25 and 50 percent of all high 
school students take these courses and tests in a given year. 

Note: After the release of a prior version of this report, we discovered a reporting issue with the OCR data for a small set of 
schools in Minneapolis. Because of this irregularity, we have dropped these schools from this indicator. The figures in this version 
of the report reflect the changes. 

Source: U.S. Department of Education’s Office of Civil Rights, Civil Rights Data Collection 2011-2012. 
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MEASURING UP 


INDICATOR I ACT/SAT Test-Taking 


In 30 of the 50 cities, schools 
report that less than 15% of all 
high school students take the 
ACT/SAT in a given year. 


d 


In 20 of the 50 cities, 


black students took 


1 


the ACT/SAT at the 
same or better rates 
than white students. 


In 17 of the 50 cities, 


Hispanic students took 
the ACT/SAT at the 




same or better rates 
than white students. 


W 
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Figure 17. Overall Share of Students Taking ACT/SAT and Percentage Point Gaps 
Between Groups, 2011-2012 
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Definitions and Sources 

The measures in Figure 17 focus on the share of students taking the ACT/SAT. The data come from the US Department 
of Education’s Office of Civil Rights’ CRDC survey. We calculated the rates by dividing the number of course/test takers 
in each high school by the total enrollment in that high school. We calculated sub-group rates by dividing the number 
of course/test takers in the subgroup in each high school by the total enrollment for that sub-group in that high school. 
Since most students take the ACT/SAT in 11th and 12th grade, we might roughly expect that somewhere between 25 and 
50 percent of all high school students take these courses and tests in a given year. For example, Memphis’s citywide 
results for the ACT/SAT (gray bars) fall in that range: just under a third of its students took the ACT/SAT in 2011-2012. 

Note: After the release of a prior version of this report, we discovered a reporting issue with the OCR data for a small set of 
schools in Minneapolis. Because of this irregularity, we have dropped these schools from this indicator. The figures in this 
version of the report reflect the changes. 

Source: U.S. Department of Education’s Office of Civil Rights, Civil Rights Data Collection 2011-2012. 
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MEASURING UP 


INDICATOR | Out-of-School Suspensions 


On average across the 50 cities, 

black students were nearly twice 
as likely as white students to 

receive out-of-school suspensions. 


I 


Baton Rouge is the only city 


where black students were not suspended 
at higher rates than white students. 
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Figure 18. Overall Share of Students Receiving Out-of-School Suspensions and 
Percentage Point Gaps Between Groups, 2011-2012 
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* We removed these cities for over-reporting or under-reporting out-of-school suspensions following the approach of the Civil Rights Project at UCLA. 


Definitions and Sources 

For Figure 18, we use school-level survey data from the U.S. Department of Education’s Office of Civil Rights’ (OCR) CRDC on 
out-of-school suspensions in 2011-2012. OCR data offers the best available information on student discipline nationwide, but 
since behaviors that warrant suspensions can differ between cities, direct comparisons between cities should be made with 
caution (note: we excluded Philadelphia, New York City, and San Francisco because of concerns about data quality, such as 
reporting zero out-of-school suspensions). We calculated the rates based on the reported number of students who received 
one or more out-of-school suspensions, and we counted students with multiple suspensions only once. We divided school 
counts by school enrollments to calculate rates for schools and then we averaged these rates across all schools in the city. 

Note: After the release of a prior version of this report ; we discovered a reporting issue with the OCR data for a small set of 
schools in Minneapolis. Because of this irregularity, we have dropped these schools from this indicator. The figures in this version 
of the report reflect the changes. 

Source: U.S. Department of Education’s Office of Civil Rights, Civil Rights Data Collection 2011-2012. 
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MEASURING UP 


APPENDIX A: DATA TABLES 


Table A1. State Data Inventory 


State 

Years covered 
(Yl, Y2, Y3) 

Proficiency categories 

Subjects analyzed 
in this report 

School-level scale 
score available 

Arizona 

2012, 2013, 2014 

Below-basic, basic, proficient, advanced, 
passing 

Math, reading 

Yes 

California 

2011, 2012, 2013 

Proficient, advanced 

Math, ELA 

Yes 

Colorado 

2012, 2013, 2014 

Below-basic, basic, proficient, advanced 

Math, reading 

No 

District of 
Columbia 

2012, 2013, 2014 

Below-basic, basic, proficient, advanced 

Math, reading 

No 

Florida 

2012, 2013, 2014 

Failing, below-basic, proficient, 
advanced, mastery 

Math, reading 

Yes 

Georgia 

2011, 2012, 2013 

Below-basic, basic, proficient, advanced 

Math, reading 

No 

Illinois 

2012, 2013, 2014 

Warning, below-basic, basic, proficient 

Math, reading 

No 

Indiana 

2011, 2012, 2013 

Passing 

Math, ELA 

No 

Kansas 

2011, 2012, 2013 

Proficient 

Math, reading 

No 

Kentucky 

2012, 2013, 2014 

Below-basic, basic, proficient, advanced 

Math, ELA, reading 

No 

Louisiana 

2012, 2013, 2014 

Unsatisfactory, below-basic, basic, 
proficient, advanced 

Math, ELA 

No 

Maryland 

2011, 2012, 2013 

Basic, proficient, advanced 

Math, reading 

No 

Massachusetts 

2012, 2013, 2014 

Failing, needing improvement, 
proficient, advanced 

Math, ELA 

No 

Michigan 

2012, 2013, 2014 

Below-basic, basic, proficient, advanced 

Math, ELA 

Yes 

Minnesota 

2012, 2013, 2014 

Below-basic, basic, proficient, advanced 

Math, reading 

Yes 

Missouri 

2012, 2013, 2014 

Level not determined, below-basic, 
basic, proficient, advanced 

Math, ELA 

Yes 

New Jersey 

2011, 2012, 2013 

Basic, proficient, advanced 

Math, ELA 

No 

New Mexico 

2012, 2013, 2014 

Below-basic, basic, proficient, advanced 

Math, reading 

Yes 

New York 

2012, 2013, 2014 

Below-basic, basic, proficient, advanced 

Math, ELA 

Yes 

North Carolina 

2011, 2012, 2013 

Below-basic, basic, proficient, advanced 

Math, reading 

Yes 

Ohio 

2012, 2013, 2014 

Proficient 

Math, reading 

No 

Oregon 

2012, 2013, 2014 

Basic, proficient, advanced 

Math, reading 

No 

Pennsylvania 

2010, 2011, 2012 

Below-basic, basic, proficient, advanced 

Math, reading 

No 

Tennessee 

2012, 2013, 2014 

Below-basic, basic, proficient, advanced 

Math, reading, 
English II, English III 

No 

Texas 

2011, 2013, 2014 

Proficient 

Math, ELA, reading 

No 

Washington 

2010, 2011, 2012 

Below-basic, basic, proficient, advanced 

Math, reading 

Yes 

Wisconsin 

2012, 2013, 2014 

Below-basic, basic, proficient, advanced 

Math, ELA, reading 

No 


Notes 

Years Covered: In Table AT, the year refers to the end of the academic school year. For example, 2014 refers to the 2013-2014 school year. 
Proficiency categories: For our analyses we collapsed proficient with higher-proficiency categories. 

Subjects: Some states refer to English Language Arts (ELA) achievement and others use the word reading. In our report, we refer to both 
types of scores as reading. 
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Table A2. The Education Landscape in the 50 Cities (2011-2012) 


City 

# 

Schools 

Total 

enroll- 

ment 

# 

Districts 

# 

Traditional 

schools 

# 

Authorizers 

Traditional 

school 

enrollment 

# 

Charter 

schools 

Charter 

school 

enrollment 

# 

Private 

schools 

# 

Private 

enrollment 

% 

White, 

public 

school 

% 

Black, 

public 

school 

% 

Hispanic, 

public 

school 

% 

FRL, 

public 

school 

Albuquerque, NM 

204 

89,131 

2 

111 

5 

71,971 

43 

8,448 

50 

8,712 

24.59 

2.74 

62.56 

59.52 

Atlanta, GA 

146 

60,846 

1 

94 

5 

44,641 

14 

4,833 

38 

11,372 

15.22 

75.32 

6.83 

73.62 

Baltimore, MD 

264 

87,960 

3 

149 

5 

63,555 

38 

12,197 

77 

12,208 

7.91 

85.51 

4.6 

83.78 

Baton Rouge, LA 

102 

43,180 

5 

57 

8 

31,399 

14 

4,282 

31 

7,499 

13.73 

78.33 

3.99 

81.64 

Boston, MA 

187 

65,825 

1 

108 

3 

49,009 

22 

6,819 

57 

9,997 

12.48 

36.13 

41.31 

69.37 

Chandler, AZ 

72 

44,843 

3 

45 

5 

38,466 

13 

4,388 

14 

1,989 

53.03 

6.86 

27.31 

31.76 

Chicago, IL 

942 

428,239 

1 

580 

3 

331,498 

38 

44,559 

324 

52,182 

8.82 

41.5 

44.5 

86.69 

Chula Vista, CA 

76 

52,144 

2 

49 

7 

44,287 

11 

5,283 

16 

2,574 

12.09 

4.2 

68.23 

13.2 

Cincinnati, OH 

121 

47,287 

1 

52 

11 

28,588 

24 

6,513 

45 

12,186 

19.91 

71.52 

2.8 

69.76 

Cleveland, OH 

186 

65,173 

1 

94 

13 

40,366 

57 

15,999 

35 

8,808 

18.51 

64.89 

12.42 

75.8 

Colorado Springs, 
CO 

155 

75,405 

5 

109 

12 

61,265 

24 

11,653 

22 

2,487 

58.7 

7.48 

24.09 

37.31 

Columbus, OH 

232 

88,044 

7 

116 

20 

53,304 

64 

24,401 

52 

10,339 

37.31 

47.68 

6.75 

59.37 

Dallas, TX 

402 

206,235 

11 

262 

14 

165,536 

56 

20,681 

84 

20,018 

6.96 

25.93 

64.23 

84.87 

Denver, CO 

225 

85,401 

2 

130 

6 

66,028 

31 

10,098 

64 

9,275 

20.61 

14.08 

58.1 

71.73 

Detroit, Ml 

230 

100,118 

1 

119 

12 

63,563 

87 

33,646 

24 

2,909 

3.69 

84.97 

10.03 

82.33 

Fort Wayne, IN 

79 

39,302 

4 

53 

6 

30,007 

3 

1,506 

23 

7,789 

48.53 

23.88 

14.2 

70 

Houston, TX 

721 

370,909 

17 

400 

23 

298,796 

138 

43,613 

183 

28,500 

10.69 

24.24 

59.41 

76.81 

Indianapolis, IN 

272 

142,608 

9 

162 

12 

113,141 

28 

11,490 

82 

17,977 

38.9 

376 

15.45 

67.16 

Jacksonville, FL 

277 

134,549 

1 

156 

3 

108,800 

18 

5,164 

103 

20,585 

37.11 

46.13 

8.57 

54.07 

Kansas City, MO 

177 

72,474 

10 

98 

16 

52,548 

35 

9,983 

44 

9,943 

34.77 

43.64 

15.69 

62.47 

Los Angeles, CA 

1,134 

582,233 

3 

546 

10 

424,830 

185 

87,271 

403 

70,132 

10.88 

9.72 

70.5 

28.8 

Louisville, KY 

187 

95,132 

1 

129 

2 

77,432 

0 

0 

58 

17,700 

49.85 

37.46 

6.99 

61.61 

Memphis, TN 

271 

120,634 

1 

176 

3 

100,236 

25 

6,310 

70 

14,088 

7.21 

82.35 

8.54 

76.5 

Mesa, AZ 

138 

81,670 

2 

84 

6 

69,400 

41 

11,305 

13 

965 

52.79 

4.58 

35.52 

50.25 

Miami, FL 

115 

48,049 

1 

55 

3 

33,368 

24 

6,839 

36 

7,842 

4.7 

27.07 

67.01 

82.69 

Milwaukee, Wl 

292 

103,696 

1 

132 

6 

60,070 

56 

16,913 

104 

26,713 

14.18 

54.77 

24.78 

83.13 

Minneapolis, MN 

139 

45,648 

2 

76 

16 

33,232 

44 

8,325 

19 

4,091 

30.45 

39.29 

16.23 

68.79 

Nashville, TN 

183 

87,107 

1 

120 

3 

70,539 

11 

2,277 

52 

14,291 

31.62 

46.66 

17.34 

71.28 

New Orleans, LA 

136 

55,764 

3 

22 

6 

8,506 

68 

31,667 

46 

15,591 

8.92 

85.19 

2.99 

82.87 

New York, NY 

2,398 

1,160,171 

1 

1,473 

5 

915,270 

135 

42,695 

790 

202,206 

14.93 

28.92 

40.34 

75.18 

Newark, NJ 

107 

44,572 

1 

67 

4 

33,371 

17 

7,814 

23 

3,387 

6.71 

56.76 

35.47 

84.95 

Oakland, CA 

183 

55,119 

1 

97 

4 

37,276 

34 

9,592 

52 

8,251 

8.81 

30.57 

40.95 

54.58 

Philadelphia, PA 

516 

221,254 

1 

240 

4 

136,305 

82 

46,628 

194 

38,321 

14.1 

58.53 

17.81 

77.62 

Phoenix, AZ 

491 

268,532 

28 

275 

32 

220,544 

139 

33,691 

77 

14,297 

29.94 

7.32 

56.08 

58.52 

Pittsburgh, PA 

110 

34,204 

1 

56 

3 

24,170 

9 

2,634 

45 

7,400 

32.88 

56.77 

1.65 

69.57 

Portland, OR 

193 

71,357 

7 

106 

12 

58,724 

11 

2,018 

76 

10,615 

52.18 

11.26 

18.78 

55.13 

Raleigh, NC 

110 

71,883 

2 

69 

4 

59,108 

9 

3,244 

32 

9,531 

42.01 

31.9 

16.09 

41.7 

Sacramento, CA 

144 

77,404 

6 

103 

10 

63,663 

16 

8,009 

25 

5,732 

15.19 

19.43 

37.18 

19.32 

San Diego, CA 

368 

199,644 

10 

242 

15 

166,906 

44 

15,541 

82 

17,197 

26.61 

8.39 

44.23 

18.67 

San Francisco, CA 

213 

76,914 

2 

101 

6 

51,098 

15 

3,490 

97 

22,326 

11.99 

10.55 

25.1 

55.55 

San Jose, CA 

288 

166,867 

19 

189 

28 

136,550 

34 

15,159 

65 

15,158 

16.56 

2.74 

46.71 

23.14 

Santa Ana, CA 

87 

66,965 

3 

62 

5 

57,858 

5 

3,751 

20 

5,356 

2.72 

0.46 

92.39 

14.1 

Seattle, WA 

166 

61,292 

2 

95 

3 

47,972 

0 

0 

71 

13,320 

43.26 

18.51 

12.4 

41.16 

St. Paul, MN 

124 

51,954 

1 

64 

14 

35,465 

33 

10,060 

27 

6,429 

27.2 

25.43 

12.96 

71.2 

Stockton, CA 

107 

60,670 

5 

70 

10 

50,906 

20 

6,930 

17 

2,834 

13.03 

13.89 

47.85 

28.49 

Tampa, FL 

136 

61,146 

1 

81 

3 

51,242 

17 

2,641 

38 

7,263 

29.16 

34.29 

28.88 

62.65 

Toledo, OH 

117 

44,883 

2 

60 

12 

28,707 

30 

7,469 

27 

8,707 

44.77 

38.88 

8.4 

64.13 

Tucson, AZ 

219 

85,711 

6 

114 

10 

67,386 

71 

13,149 

34 

5,176 

24.55 

5.32 

62.37 

44.71 

Washington, DC 

285 

74,214 

1 

111 

6 

37,277 

101 

22,936 

73 

14,001 

8.48 

74.92 

13.67 

60.2 

Wichita, KS 

114 

56,969 

5 

88 

6 

48,783 

0 

0 

26 

8,186 

38.37 

17.46 

29.91 

72.18 


Notes 

These data are for the 2011-2012 school year. The number of districts in a city reflects the count of the number of unique districts in each city, and the number of charter 
authorizers is a count of the number of unique charter authorizers. Agencies that oversee both traditional schools and charter schools may appear in each measure. 
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Table A3. Changing Population Demographics in the 50 Cities 


City 

Total population 

White population 

Black population 

Hispanic population 

Asian population 

2000 

2013 

% A 

2000 

2013 

% A 

2000 

2013 

% A 

2000 

2013 

% A 

2000 

2013 

% A 

Albuquerque, NM 

473,729 

556,489 

17.47 

239,837 

235,427 

- 1.84 

15,238 

20,536 

34.77 

189,075 

263,196 

39.2 

11,632 

17,218 

48.02 

Atlanta, GA 

433,364 

447,848 

3.34 

132,100 

166,290 

25.88 

271,974 

233,453 

- 14.16 

18,632 

24,004 

28.83 

8,901 

20,889 

134.68 

Baltimore, MD 

651,154 

622,104 

- 4.46 

204,200 

176,691 

- 13.47 

420,811 

394,132 

- 6.34 

11,101 

28,440 

156.19 

11,392 

17,191 

50.9 

Baton Rouge, LA 

247,290 

229,405 

- 7.23 

118,360 

82,121 

- 30.62 

116,068 

128,637 

10.83 

5,043 

6,230 

23.54 

6,871 

9,711 

41.33 

Boston, MA 

603,388 

644,710 

6.85 

308,480 

297,158 

- 3.67 

151,510 

157,777 

4.14 

85,859 

121,496 

41.51 

47,033 

61,697 

31.18 

Chandler, AZ 

180,022 

249,139 

38.39 

125,049 

149,683 

19.7 

6,734 

15,181 

125.44 

37,536 

55,810 

48.68 

8,538 

26,579 

211.3 

Chicago, IL 

2 , 895,409 

2 , 718,789 

- 6.1 

928,974 

877,092 

- 5.58 

1 , 100,000 

870,752 

- 18.23 

753,751 

782,617 

3.83 

137,433 

177,697 

29.3 

Chula Vista, CA 

179,862 

256,765 

42.76 

59,682 

46,316 

- 22.4 

8,653 

10,860 

25.51 

88,015 

160,801 

82.7 

21,491 

36,315 

68.98 

Cincinnati, OH 

344,107 

297,498 

- 13.54 

186,759 

149,710 

- 19.84 

145,485 

128,929 

- 11.38 

4,225 

9,924 

134.89 

5,743 

6,224 

8.38 

Cleveland, OH 

478,600 

390,106 

- 18.49 

189,473 

132,768 

- 29.93 

244,678 

206,587 

- 15.57 

34,549 

41,976 

21.5 

7,628 

7,294 

- 4.38 

Colorado Springs, 
CO 

382,234 

439,858 

15.08 

291,361 

309,760 

6.31 

27,553 

31,405 

13.98 

45,550 

75,393 

65.52 

13,739 

18,110 

31.81 

Columbus, OH 

756,172 

822,762 

8.81 

521,287 

484,234 

- 7.11 

183,649 

237,043 

29.07 

19,424 

49,851 

156.65 

27,681 

39,406 

42.36 

Dallas, TX 

1 , 186,496 

1 , 257,676 

6 

415,611 

375,952 

- 9.54 

309,081 

309,699 

0.2 

421,611 

519,736 

23.27 

35,099 

46,691 

33.03 

Denver, CO 

554,603 

649,495 

17.11 

291,693 

350,602 

20.2 

63,883 

66,380 

3.91 

176,039 

200,805 

14.07 

17,417 

26,412 

51.64 

Detroit, Ml 

955,916 

688,740 

- 27.95 

110,828 

62,165 

- 43.91 

781,642 

558,087 

- 28.6 

47,317 

53,300 

12.64 

11,296 

10,193 

- 9.76 

Fort Wayne, IN 

274,834 

251,340 

- 8.55 

216,328 

173,751 

- 19.68 

38,948 

44,254 

13.62 

13,190 

21,064 

59.7 

4,920 

9,213 

87.26 

Houston, TX 

2 , 595,962 

2 , 197,374 

- 15.35 

958,246 

573,334 

- 40.17 

581,686 

510,171 

- 12.29 

888,228 

960,558 

8.14 

157,164 

149,195 

- 5.07 

Indianapolis, IN 

783,941 

838,425 

6.95 

533,403 

481,281 

- 9.77 

204,064 

244,444 

19.79 

29,725 

83,427 

180.66 

12,270 

23,171 

88.84 

Jacksonville, FL 

733,607 

842,588 

14.86 

460,618 

460,261 

- 0.08 

215,317 

264,854 

23.01 

30,117 

68,727 

128.2 

23,077 

43,704 

89.38 

Kansas City, MO 

453,311 

467,082 

3.04 

269,851 

261,240 

- 3.19 

139,822 

136,099 

- 2.66 

30,612 

48,562 

58.64 

9,791 

14,584 

48.95 

Los Angeles, CA 

3 , 740,133 

3 , 884,340 

3.86 

1 , 172,810 

1 , 108,784 

- 5.46 

421,722 

358,536 

- 14.98 

1 , 720,739 

1 , 913,501 

11.2 

399,506 

471,672 

18.06 

Louisville, KY 

579,826 

609,908 

5.19 

434,578 

414,179 

- 4.69 

124,344 

146,030 

17.44 

9,686 

29,740 

207.04 

8,579 

15,865 

84.93 

Memphis, TN 

732,608 

653,450 

- 10.8 

279,088 

179,954 

- 35.52 

418,039 

415,155 

- 0.69 

19,910 

43,373 

117.85 

13,278 

12,616 

- 4.99 

Mesa, AZ 

440,867 

457,595 

3.79 

326,143 

292,307 

- 10.37 

11,565 

18,080 

56.33 

83,825 

123,972 

47.89 

8,128 

11,112 

36.71 

Miami, FL 

374,798 

417,670 

11.44 

47,581 

41,422 

- 12.94 

77,588 

74,559 

- 3.9 

246,104 

296,926 

20.65 

2,487 

3,121 

25.49 

Milwaukee, Wl 

596,854 

599,168 

0.39 

275,776 

224,654 

- 18.54 

225,701 

242,144 

7.29 

70,992 

104,627 

47.38 

17,867 

23,499 

31.52 

Minneapolis, MN 

382,465 

400,079 

4.61 

242,634 

244,126 

0.61 

76,088 

81,692 

7.37 

29,085 

38,624 

32.8 

26,454 

27,157 

2.66 

Nashville, TN 

544,538 

634,465 

16.51 

352,389 

358,699 

1.79 

149,777 

183,398 

22.45 

25,397 

64,932 

155.67 

13,808 

23,896 

73.06 

New Orleans, LA 

484,674 

378,715 

- 21.86 

130,707 

118,490 

- 9.35 

325,424 

225,257 

- 30.78 

15,032 

20,849 

38.7 

11,293 

12,034 

6.56 

New York, NY 

8 , 008,278 

8 , 405,837 

4.96 

2 , 870,205 

2 , 753,018 

- 4.08 

2 , 000,000 

1 , 959,076 

- 4.32 

2 , 161,530 

2 , 428,756 

12.36 

847,587 

1 , 173,030 

38.4 

Newark, NJ 

273,115 

278,436 

1.95 

41,911 

29,414 

- 29.82 

144,687 

139,613 

- 3.51 

80,279 

102,582 

27.78 

3,507 

2,779 

- 20.76 

Oakland, CA 

396,638 

406,228 

2.42 

96,229 

108,415 

12.66 

144,430 

109,241 

- 24.36 

86,544 

108,081 

24.89 

63,856 

74,130 

16.09 

Philadelphia, PA 

1 , 517,550 

1 , 553,165 

2.35 

653,240 

565,546 

- 13.42 

658,728 

666,344 

1.16 

128,300 

206,033 

60.59 

70,220 

109,779 

56.34 

Phoenix, AZ 

1 , 331,131 

1 , 513,350 

13.69 

753,122 

689,708 

- 8.42 

68,833 

113,299 

64.6 

453,386 

624,916 

37.83 

29,647 

54,187 

82.77 

Pittsburgh, PA 

334,524 

305,838 

- 8.58 

225,611 

202,282 

- 10.34 

92,395 

76,815 

- 16.86 

4,514 

8,216 

82.01 

9,869 

16,279 

64.95 

Portland, OR 

546,950 

611,134 

11.73 

421,394 

444,576 

5.5 

39,754 

41,493 

4.37 

37,626 

60,974 

62.05 

39,439 

54,119 

37.22 

Raleigh, NC 

325,013 

431,897 

32.89 

206,824 

228,077 

10.28 

83,427 

129,681 

55.44 

21,174 

46,387 

119.08 

11,730 

21,999 

87.54 

Sacramento, CA 

403,496 

479,671 

18.88 

169,581 

171,567 

1.17 

65,857 

71,508 

8.58 

86,849 

128,472 

47.93 

72,934 

95,971 

31.59 

San Diego, CA 

1 , 278,876 

1 , 355,885 

6.02 

651,806 

589,241 

- 9.6 

106,672 

90,903 

- 14.78 

320,224 

405,789 

26.72 

185,037 

257,807 

39.33 

San Francisco, CA 

776,733 

837,442 

7.82 

345,686 

350,032 

1.26 

63,407 

50,078 

- 21.02 

109,565 

128,395 

17.19 

249,887 

294,308 

17.78 

San Jose, CA 

931,917 

998,514 

7.15 

350,732 

278,164 

- 20.69 

33,525 

34,196 

2 

282,860 

334,735 

18.34 

255,681 

345,643 

35.19 

Santa Ana, CA 

329,820 

334,241 

1.34 

41,311 

36,419 

- 11.84 

4,400 

2,759 

- 37.3 

250,846 

259,230 

3.34 

31,023 

34,826 

12.26 

Seattle, WA 

596,104 

652,429 

9.45 

414,237 

442,438 

6.81 

55,242 

53,934 

- 2.37 

31,418 

41,990 

33.65 

85,415 

103,019 

20.61 

St. Paul, MN 

287,144 

294,873 

2.69 

186,174 

160,171 

- 13.97 

36,613 

53,054 

44.9 

22,693 

28,229 

24.4 

38,225 

51,169 

33.86 

Stockton, CA 

267,710 

298,115 

11.36 

94,036 

66,919 

- 28.84 

27,941 

35,973 

28.75 

88,995 

129,855 

45.91 

53,463 

66,725 

24.81 

Tampa, FL 

313,179 

352,981 

12.71 

164,041 

162,746 

- 0.79 

78,287 

86,266 

10.19 

61,220 

83,677 

36.68 

7,522 

16,191 

115.25 

Toledo, OH 

328,951 

282,313 

- 14.18 

229,003 

174,870 

- 23.64 

76,752 

80,059 

4.31 

17,622 

22,084 

25.32 

4,084 

4,083 

- 0.02 

Tucson, AZ 

496,411 

526,141 

5.99 

272,131 

241,153 

- 11.38 

21,870 

31,011 

41.8 

178,852 

222,400 

24.35 

13,653 

20,820 

52.49 

Washington, DC 

572,059 

646,449 

13 

160,525 

232,279 

44.7 

346,354 

315,559 

- 8.89 

45,015 

65,560 

45.64 

16,565 

28,118 

69.74 

Wichita, KS 

373,597 

386,558 

3.47 

277,587 

247,580 

- 10.81 

42,480 

50,489 

18.85 

33,729 

63,684 

88.81 

15,017 

21,209 

41.23 
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Notes 

Decennial census surveys were used for 2000. We use data from the 2013 American Community Survey one-year estimates for population 
numbers in 2013. These data are available at the place-level (e.g., “Chicago city”). Data at the place-level and for racial/ethnic group counts 
are not available for 2014 and 2015. Starting in 2000, the Census allowed survey respondents to select more than one racial category, 
introducing multiracial categories. We use race-bridging variables to compare racial and ethnic group counts between 2000 and 2013. 
Race-bridging variables are created by reassigning multiracial counts to single-race groups. We create new racial-ethnic group variables by 
adopting the method used in the GeoLytics Neighborhood Change Database, previously developed at the Urban Institute by Jeffrey Passel. 
See the GeoLytics Data Users Guide for more information. White, black, and Asian population counts exclude respondents who identify as 
Hispanic/Latino. In other words, these groups reflect non-Hispanic/Latino whites, blacks, and Asians. Percent change is calculated as (2013 - 
2000 )/ 2000 * 100 . 
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APPENDIX B: HOW WE MEASURED THE INDICATORS 


Measuring Citywide Gains in Math and Reading Proficiency 

For each city, we calculated the average gains made in math and reading proficiency over the three-year period for which we 
have data. Specifically, we estimated a separate linear regression model for each city in our sample, in which the outcome 
variable is the mean-centered (by state and year) proportion of students in a school scoring at or above "proficient” on each 
state’s standardized math test (we also did the same for reading). The model can be expressed as follows: 

VA + A (Yea/y + R{( jt + (i 3 S. t + £, 

where Y. is the dependent variable (the within-state-year standardized proportion of students in school / at time f scoring 
at or above “proficient” in math or reading), Year t represents the year as a linear term (1 = year 1, 2 = year 2, 3 = year 3), 

X. ( is a vector representing a school’s student composition (the proportion FRL, white, black, Hispanic and Asian) S Jt is a 
vector representing other school characteristics (whether the school is in an urban area, the grade level of the school, and 
the school’s total enrollment) and £ is the residual error. We exclude schools with missing information on these variables. 
The coefficient of primary interest here is R r which tells us the average change in performance per year for the schools in a 
given city. To approximate the change over two years (from year one to year three), we multiply R 1 by 2, yielding the results 
presented in Figure 4. Positive values indicate gains made by schools in a city, while negative values indicate losses. 

To account for the fact that schools will show up multiple times in our data over the three years, we used clustered standard 
errors to compute confidence intervals. If the 95 percent confidence interval includes zero, then the gain (or loss) is not 
statistically significant and the bars in the figure are not shaded. If the 95 percent confidence interval does not contain zero, 
then the gain (or loss) is statistically significant and the city’s bar in the figure is shaded. 

We also estimated a series of simple growth models in which we allowed the intercepts to vary randomly among schools; 
results were substantively similar to those presented in the report. 


Measuring High School Graduation Rates 

For each city we calculated graduation rates using the following expression: 

Number of cohort members who earned a regular high school diploma by the end of the 2012-13 school year 


Number of first-time 9th graders in fall 2009 (starting cohort) plus students who transferred in, minus students who 
transferred out, emigrated, or died during school years 2009-10, 2010-11, 2011-12, and 2012-13 

When graduation rates were expressed as a range due to suppression (e.g., 50-54 percent), the midpoint for that range was 
used as the rate (e.g., 52.5 percent). 


Measuring Share of Students in Schools That Are “Beating the Odds ” 

We used the proportion of students in each school scoring at or above “proficient” on the state’s standardized test in math 
as the outcome measure, and the school’s demographic measures as predictors in a series of multivariate linear regression 
models. We estimate separate models for each state and each year. These models can be expressed by the following 
equation: 


Yj = R 0 + R {White), + R 2 (Black ) , + R 3 (Hispanic) j + R 4 (FRL) j + R s (ELL) j 
+ R 5 (Urban) j + R 6 (SchoolLevel) ] + R r (SchoolSize) j + £ 
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where Y is the dependent variable (the proportion of students in school j scoring at or above “proficient”), controlling for the 
proportion of students who are white, black, Hispanic, eligible for FRL, and English language learners, as well as for whether 
the school is in an urban area, its highest grade level (a set of dummy variables indicating whether highest grade is in middle 
school [6-8], high school [9-12], and ungraded/missing, with elementary school [K-5] as the reference category), and its size 
(i.e., total enrollment). We exclude schools with missing information on these variables. 

The school-level residual, £ , indicates whether a school outperformed similar schools statewide. Schools are said to have 
performed better than expected if their residuals are positive (their observed proficiency rate is greater than their expected 
rate), while schools are said to have performed worse than expected if their residuals are negative (their observed proficiency 
rate is less than their expected rate). We used the standard error of the residual to calculate a 90 percent confidence interval 
around each school’s residual as a measure of the residual’s precision using the formula below: 

90% Cl = school residual ± 1.645 x residual standard error 

If the confidence interval contains 0, then whether the school was outperforming similar schools statewide cannot be 
determined with reliability. Therefore, schools are considered to be outperforming similar schools statewide only if their 
residuals were positive and their residuals’ confidence interval did not contain zero. 

We then calculated the percentage of students in each city in each year that attended schools that were outperforming 
similar schools statewide according to our definition. 


Measuring Enrollment in High- and Low-Scoring Elementary and Middle Schools 

By city, year, and school level, we identified the 20th and 80th percentile in terms of percent of students scoring proficient 
on the reading and math tests. We then calculated the percentage of students in the city that fell into the bottom and top 20 
percent of schools. We did this for overall enrollment, as well as for students in particular subgroups: FRL vs. non-FRL, and 
white vs. black vs. Hispanic. 

In these analyses, we focus on elementary and middle schools because proficiency data is widely available for 3rd through 
8th grades. To identify school level, we used the NCES CCD definition to construct two categories: (1) schools classified as 
primary (low grade: Pre-K through 03; high grade: Pre-K through 08) or middle (low grade: 04 through 07; high grade: 04 
through 11) schools; and (2) schools classified as high schools (low grade: 07 through 12; high grade: 12). We excluded schools 
classified by the CCD as “other” (any other configuration not falling within the above three categories, including ungraded 
and operational schools with non-applicable grade spans). Because not all state datasets include high school test scores, we 
can report only the primary and middle school results. 
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ENDNOTES 


1. For background on the problems facing Detroit’s education system, see Robin Lake, Ashley Jochim, and Michael 
DeArmond, “ Fixing Detroit’s Broken School System .” Education Next 15, no. l(Winter 2015): 20-27. For more on Hratchian 
and others’ work on rebuilding Detroit, see The Choice is Ours (Detroit, Ml: Coalition for the Future of Detroit Schoolchildren, 
2015); Rochelle Riley, “ Detroit Commission Reaches Consensus or School Governance ," Detroit Free Press, Feb. 27, 2015. 

2. In most places, state education agencies have longitudinal data systems that link information about students’ academic 
achievement with information on their teachers, schools, and districts (for example, see Data for Action 2014 (Washington, 

DC: Data Quality Campaign, 2014). These data allow city leaders to pull individual reports on schools and districts, but 
state data systems do not typically provide a way to pull city-level reports. If a leader in Memphis wanted to know how all 
of her city’s schools were doing, for example, she would need to pull reports on a subset of schools that are part of the 
Shelby County Schools and the Tennessee’s Achievement School District and then piece together a citywide summary. If 
that same leader, like Hratchian, wanted to see how her city’s schools compared to other places— perhaps she is especially 
concerned about the prospects for Memphis’s short- and long-term economic development— she would also be out of luck. 

As Hratchian’s story shows, the best available tool for cross-city comparisons, the U.S. Department of Education’s Trial Urban 
District Assessment, or TUDA, includes only school districts (see The Nation’s Report Card, TriaiUrban District Assessment 
www.nationsreportcard.gov . for a list of the 21 cities in the NAEP TUDA). Today’s educational landscape is more complex 
than that. Multiple agencies oversee public schools in cities like Memphis, multiple school districts operate in cities like 
Phoenix, and large charter sectors operate in cities like Cleveland. Leaders have very few places to turn if they want a holistic 
comparison of their schools and schools in other cities. This is not to say state data systems are not important. Researchers 
are using them to rigorously study important topics ranging from teacher quality to school accountability. Studies done by 
researchers at the American Institutes for Research’s National Center for Analysis of Longitudinal Data in Education Research 
( CALDER ) illustrate the importance of this work. Nevertheless, these studies tend to focus on high-profile school districts or 
entire states, rather than the public school options that families actually have in a given city. These rigorous studies also rarely 
compare cases across city or state boundaries. 

3. For example, see Grover J. Whitehurst and Ellie Klein, The 2014 Education Choice and Competition Index (Washington, DC: 
The Brookings Institution, 2014); George W. Bush Institute, Mayor’s Report Card on Education (Dallas, TX: George W. Bush 
Institute, 2015). 

4. For example, see William H. Frey, Diversity Explosion: How New Racial Demographics Are Remaking America (Washington, 
DC: Brookings Institution Press, 2014); Plussar and Bailey, Projections of Education Statistics to 2022 : Maxwell, “ US School 
Enrollment Hits Majority-Minority Milestone .” Education Week ; Suitts, A New Majority: Low-Income Students Now a Majority in 
the jjailib's ScMotS : and NCES, Digest of Education Statistics: 2013 . Table 203.50. 

5. Private schools make up a non-trivial share of enrollments in several cities; however, we were unable to include any 
information on private schools for the indicators because we do not have any performance data for private schools. 

6. Specifically, we used the boundaries of “incorporated places” from the U.S. Census. 

7. More specifically, we calculated a Herfindahl index to identify difference in competition in the cities (technically, the sum of 
the squares of each sector’s market shares). The enrollment distributions we used to calculate the index clearly understate the 
amount of choice parents have in each city because the data don’t capture whether parents have choices within or between 
traditional school districts, whether they exercise choice by choosing their residence, or whether they use public vouchers to 
attend private schools. After sorting by the Herfindahl index we then sorted the cities by total enrollments and selected the 
top 50. 

8. All but three of our cities— Seattle, Louisville, and Wichita— offered charter schools during the 2011-12 school year. 
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9. For example, see Gary Phillips and Alicia Garcia, Aiming High: Setting Performance Standards for Student Success 
(Washington DC: Education Policy Center at American Institutes for Research, 2015): 1-12; Douglas N. Harris, Value-Added 
Measures in Education: What Every Educator Needs to Know (Cambridge, MA: Plarvard Education Press, 2011); Institute of 
Education Sciences, Mapping State Proficiency Standards Onto the NAEP Scales: Variation and Change in State Standards for 
Reading and Math, 2005-2009 (Washington, DC: National Center for Education Statistics, US Department of Education, 2011). 

10. States could and should easily fix the problems associated with proficiency data by making aggregate school-level scale 
scores publicly available (as a handful of states already do). In the meantime, however, our indicators rely on aggregate 
data and necessarily trade off some precision in favor of coverage and transparency. For an excellent discussion of these 
issues see Robin T. Jacob, Robert D. Goddard, and Eun Sook Kim, “Assessing the Use of Aggregate Data in the Evaluation of 
School-Based Interventions: Implications for Evaluation Research and State Policy Regarding Public-Use Data,” Educational 
Evaluation and Policy Analysis 36, no. 1 (May 31, 2013): 44-66. 

11. Harris, Value-Added Measures in Education: Jal Mehta, The Allure of Order: High Hopes, Dashed Expectations, and the 
Troubled Quest to Remake American Schooling (New York, NY: Oxford University Press, 2013). 

12. Institute of Education Sciences, Mapping State Proficiency Standards Onto the NAEP Scales. 

13. These courses include trigonometry, trigonometry/algebra, trigonometry/analytic geometry, trigonometry/math analysis, 
analytic geometry, math analysis, math analysis/analytic geometry, probability and statistics, and pre-calculus. According 

to OCR, “trigonometry courses prepare students for eventual work in calculus, and typically include the following topics: 
trigonometric and circular functions; their inverses and graphs; relations among the parts of a triangle; trigonometric 
identities and equations; solutions of right and oblique triangles; and complex numbers. Analytic geometry courses include 
the study of the nature and intersection of lines and planes in space. Math analysis courses include the study of polynomial, 
logarithmic, exponential, and rational functions and their graphs; vectors; set theory; Boolean algebra and symbolic logic; 
mathematical induction; matrix algebra; sequences and series; and limits and continuity. Probability and statistics courses 
introduce the study of likely events and the analysis, interpretation, and presentation of quantitative data. Pre-calculus 
courses combine the study of trigonometry, elementary functions, analytic geometry, and math analysis topics as preparation 
for calculus.” Office of Civil Rights, personal communication with the author, June 28, 2015. 
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